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(57) The two sides of a flat panel are inspected in successive inspection stations each of which includes a 
camera having a linear anray of CCD elements which generates an image line-by-line from a stripe of 
light reflected by one flat surface of the panel as the panel is conveyed through the inspection station. 
The signals representing images of the two faces of the panel are independently digitized and stored in 
separate pixel arrays of gray-scale values. The images are queued for filtering followed by processing by 
a single image processing board which generates a table of hits, identified as pixels having gray-scale 
values which breach preselected thresholds. Sections of the stored pixel array are transmitted 
sequentially to the image processing board which has a frame size smaller than the stored array, with 
the frame sized sections overlapping as required to confine the pixels processed to those inside the 
edges of the panel image. A microprocessor analyzes the hits identified by the image processing board 
and generates a reject signal if the density of hits exceeds a limit The microprocessor also associates 
hits by proximity into blobs and rejects the panel if a weighted count of hits in any blob exceeds 
predetermined criteria. 
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This invention relates to systems for visually inspecting flat reflective parts such as metal panels for defects, 
which may include scratches, gouges, holes inclusions and such, by scanning the panel with a stripe of light, 
recording the gray-scale values of reflected light for an array of pixels covering the panel, and processing the 
gray-scale values for the array of pixels using high speed vision processing equipment to identify the defects. 

5 More particularly, the invention is directed to such systems in which modular vision processing boards with 
limited area coverage are used to inspect larger panels on-line and in which the data indicative of a defect is 
reduced to speed processing. 

Many flat parts such as for example copper laminate panels used in making printed circuit boards must be 
inspected after manufacture to identify defective units. Typical defects in these copper laminate panels include 

10 scratches, gouges, dents over certain dimensions, pin-point holes and inclusions, any of which could impair 
the performance of a printed circuit board using such a panel. The presence of a few minor scratches, dents, 
et cetera may be tolerated, however, a large number of even such minor blemishes is cause for rejection. To 
date, the inspection of such panels has been accomplished by workers visually scanning each panel. Thus, 
inspection has been labor intensive and, therefore expensive. It is also prone to variability from one inspector 

15 to another. 

It is, therefore, an object of the invention to provide a reliable, reasonably priced automated system for 
inspecting panels and other flat reflective parts. 

Accordingly, the present invention resides in a vision system for inspecting a part having a substantially 
flat reflective surface, characterized in that said system comprises an optical data acquisition system, compris- 

20 ing a light source for directing light at the substantially reflective surface; and a camera for generating pixel 
gray-scale values for light from said light source means reflected from said flat reflective surface; and a vision 
processor comprising a storage system for storing a pixel array of gray-scale values generated by said camera; 
an image processing board identifying pixels in said pixel array with predetermined gray-scale values; and a 
processor for evaluating said identified pixels and generating an output based thereon. 

25 In inspecting parts having substantially flat reflective surfaces, the vision system is adapted to evaluate 
on-line a pixel array of gray-scale values of light reflected from the reflected surfaces of the part to detect the 
presence of various defects such as scratches, gouges, holes, inclusions and such. In use, a vision system of 
one embodiment compares the gray-scale values of the pixel array to threshold values by an image processing 
board which generates a table of hits identifying the pixels having gray-scale values which breach the 

30 thresholds. The gray-scale values are filtered before the evaluation to increase contrast which reduces the num- 
ber of hits and therefore the amount of processing required. In addition to rejecting a part having a total number 
of hits which exceeds a selected value, the system associates hits by proximity into blobs and also rejects a 
part having blobs containing more than a selected number of hits. In the preferred embodiment of the invention, 
the image processing board compares the pixel gray-scale values to multiple thresholds and rejects a part 

35 based upon a weighted count of hits in a blob. 

As another aspect of the invention, the pixel processing board has a frame size smaller than the portion 
of the pixel array covering the flat reflective surface. In accordance with the invention, successive sections of 
the pixel array covering the flat reflective surface are transferred to the pixel processing board until the entire 
image of the surface has been processed. In order to reduce the number of hits and therefore the processing 

40 burden, only gray-scale values for pixels within the flat reflective surface are passed to the pixel processing 
board for processing. In the exemplary system, only the area of the surface more than one quarter inch in from 
the edges of the part is evaluated. Where the frame size of the pixel processing board is not a whole multiple 
of the portion of the pixel array covering the flat reflective surface, the frame sized sections of the array trans- 
ferred to the pixel processing board are overlapped to maintain the total coverage processed to that of the flat 

45 reflective surface. Redundant hits in the overlapping portions of the frame sized sections of the pixel array are 
eliminated. 

As applied to inspecting panels having two flat reflective surfaces, the panels are passed through two suc- 
cessive inspection stations each generating a set of pixel gray-scale values for one flat reflective surface. Sepa- 
rate pixel arrays for each surface are generated independently. The images are queued for sequential proces- 

50 sing by a single pixel processing board. Since each pixel array is larger than the frame size of the pixel pro- 
cessing board, and therefore must be processed in sections, the features of the invention, such as only pro- 
cessing the portion of the reflective surfaces more than a certain distance from the edges, overlapping the frame 
sized sections to remain within those borders, filtering of the raw gray-scale values, and associating hits into 
btobs, all contribute to reducing the processing burden so the inspection can be made on-line with reasonably 

55 priced equipment 

In order that the invention can be more clearly understood, convenient embodiments thereof will now be 
described, by way of example, with reference to the accompanying drawings in which: 
Figure 1 is a general schematic diagram illustrating a system for inspecting flat panels. 
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Figure 2 is a schematic diagram of a vision processor which forms part of the system illustrated in Figure 

1. 

Figure 3 is a schematic diagram of image pipelines which form part of the vision processor of Figure 2. 
Figure 4 is an illustration of the manner in which the image pipelines illustrated in Figure 3 process data 
5 for a large panel using modular units having a smaller frame size. 

Figures 5 through 8 and 9a and 9b are flow charts of a suitable program used by the system illustrated in 
Figures 1-3, and 

Figures 10-12 are diagrams of arrays utilized by the programs described in Figures 5 through 8 and 9a 
and 9b. 

10 Referring to the drawings, the invention will be described as applied to a system 1 for inspecting copper 
laminate panels used for making printed circuit boards. However, it will become clear to those skilled in the art 
that the invention has broad application to the inspection of a wide variety of flat parts in which defects or other 
features to be detected have a reflectivity detectabiy different from the remainder of the flat surface of the part. 
The system 1 includes two inspection stations 3a and 3b, one for inspecting each face of the copper lami- 

15 nate panels 5. An inverting transfer station 7 turns the panels 5 over after a first side has been inspected at 
the station 3a for inspection of the second face at the station 3b. 

Each of the inspection stations 3a and 3b includes a belt conveyor 9a and 9b, respectively. The panels 5 
are placed on the conveyor belt 9a by a feeder apparatus (not shown) and transferred from the conveyor belt 
9a to the conveyor belt 9b by the inverting transfer station 7. The inverting transfer station 7, which is known, 

20 includes a number of axially displaced disks 1 1 which are interieafed with roller sections of a roller conveyor 
13. The disks 1 1 have a number of angularly displaced slots 15 which receive the panels 5 from the conveyor 
9a. The disks 1 1 are indexed in sychronism with operation of the conveyors 9a and 9b to invert the panels while 
transferring them from the first inspection station 3a to the second inspection station 3b. 

As a panel 5 is carried along by the conveyor belt 9a or 9b, the leading edge of the panel is detected by a 

25 sensor 17a or 17b, respectively. In the exemplary system, optosensors are used for the sensors 17a and 17b. 
The conveyors 9a and 9b advance the panels 5 past optical data acquisition systems 19a and 19b. These opti- 
cal acquisition systems include fluorescent strip lamps 21a and 21b which direct stripes of light onto the moving 
panels oriented transversely to the direction of travel of the panels. The stripes of light strike the panels at an 
angle of incidence which is less than 90° and are reflected to cameras 23a and 23b. Each camera has a single 

30 row of 2048 CCD (charge coupled device) elements focused on an area extending the full width of the panel 
5 by 12 mils in the direction of travel. As a panel 5 traverses a camera's line-of-sight, the video information for 
an image of the panel is built up line-by-line. This video information is digitized and stored in a large memory 
in a vision processor housed in a cabinet 27. The cabinet 27 also houses a video monitor 29, a data entry ter- 
minal 31 for operator interface, and a printer 33 for generating hard copies of defect reports generated by the 

35 system. 

Each of the inspection stations 3a and 3b also includes an optical encoder 35a and 35b attached to the 
drive roll of the associated conveyor 9a and 9b, respectively. These optical encoders are used to measure belt 
speed indirectly through shaft position. As the leading edge of a panel 5 passes under the optosensor 17a or 
17b which is installed at a fixed distance in front of the line-of-view of the camera 23a or 23b, a pulse is gen- 

40 erated. An encoder reading is taken immediately after the panel triggers the optosensor. A second reading is 
taken at a subsequent time when the panel edge is near the viewline but not past The difference in encoder 
readings is proportional to the average speed for the time of travel. From the computed speed and position, 
time of arrival of the panel at the view line is computed. At that time, the video processor 25 is enabled to begin 
inputting data from the camera 23a or 23b. 

45 A block diagram of the video processor 25 which is housed in the cabinet 27 is illustrated in Figure 2. The 
various components of the system are interconnected by a VME bus 37. The processor includes a microcom- 
puter 39, the data entry terminal 31 and the printer 33, a hard disk memory 41 and a floppy disk memory 43 
and their controller 45, an encoder interface 44 which provides inputs from the optical encoders 35A and 35B, 
and a parallel input/output board 47 which also performs timing functions. The input/output board 47 interfaces 

so with an optoisolator panel 49 which provides isolated signals from the optosensors 17a and 17b. The 
input/output board 47 also provides through the optoisolator panel 49 contact outputs for alarms generated by 
the system and signal for controlling a reject station (not shown) which diverts rejected panels downstream of 
the conveyor 9b. 

The video processor 25 also includes image processing pipelines 51. These image processing pipelines 
55 control the cameras 23a and 23b and receive video signals from these cameras through camera controllers 
53a and 53b. As will be discussed, the image processing pipelines 51 process the raw video data for trans- 
mission over the VME bus 37 to the microprocessor 39. The image processing pipelines also supply video sig- 
nals to the monitor 29. 
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The image processing pipelines 51 are illustrated in block diagram form in Figure 3. These pipelines include 
a number of special purpose processors, all residing on the VME bus 37. In the exemplary system, the special 
purpose processors are all image processing boards manufactured by Data Cube, Inc. Each of the cameras 
23a and 23b sweep the elements of the linear array at a rate of 1 000 lines per second to generate analog eleo 

5 trical signals identified in Figure 3 as channel zero and channel 1 signals, respectively. These analog video 
signals are applied to line scan interface boards (MAX-SCAN) 55a and 55b which digitize the analog signals 
for storage in Capture ROI (region of interest) modules 57a and 57b. Each of the capture ROI boards 57a and 
57b has a memory array which is 2000 by 4000 pixels with 8-bit resolution. Since the largest panel inspected 
by the system generates data for an array 2000 pixels by 1500, each of the Capture ROI boards 57a and 57b 

10 can store data for two separate panels. Thus, while data is being read out for a previously imaged panel, data 
can be read in for the next panel. 

The images captured by the two cameras are queued for transmission over a line 61 in RS-170 video format 
to a MAX-SI GM A board 63 which is configured as a Laplacian filter. The filtered output of the MAX-SI GMA board 
63 is transmitted over the line 65 also in RS-1 70 video format to a FEATUREMAX board 67 which is a multilevel 

15 threshold detector. The FEATUREMAX board can compare the filtered gray-scale values to a number of posi- 
tive and negative thresholds. Since the defects to be detected have a lower reflectivity than the unblemished 
portions of the panels, the negative thresholds are used. The FEATUREMAX board 67 generates a table of 
hits identifying the pixels having gray-scale values below the selected negative thresholds. The hit table records 
the x, y coordinates of the hit and includes for each hit two bits indicating the lowest threshold breached. The 

20 Laplacian filter function applied by the MAX SIGMA board 63 enhances the contrast of the gray-scale values 
of the captured image, and therefore enhances the definition of the edges of the defects. The thresholds applied 
by the FEATUREMAX board 67 are then selected to detect these edge features which reduces the number of 
hits and therefore the amount of data that must be processed. The table of hits generated by the FEATUREMAX 
board provides the basis upon which the acceptance/rejection decision is made by the microprocessor 39. 

25 For debugging purposes, a Display ROI board 69 selects among the raw gray scale data on lines 61 from 
a selected one of the Capture ROI boards 57a or 57b, or the filtered output of the MAX SIGMA board 63 on 
the lines 65 and sends the resulting image to a MAX-GRAPH board 71 over lines 73. The MAX-GRAPH board 
71 can superimpose over the image received from the Display ROI the processed hit data received over the 
VME bus 37 from the digital processor 39. The composite image is then passed over lines 75 to the display 

30 monitor 29. 

A novel feature of the invention lies in the implementation of the threshold processing by the FEATURE- 
MAX board 67. The size and desired resolution of the image dictates that the images of the panels 5 require 
as many as approximately 2000 pixels in the cross web direction and approximately 1500 pixels in the down 
web direction for the largest panel which is 24 inches by 18 inches. The FEATUREMAX board 67, however, is 

35 limited to processing frames of 512 cross web pixels by 483 down web pixels. In order to process the entire 
panel image with the desired resolution, the FEATUREMAX board 67 is used to successively process a mul- 
tiplicity of frames covering the panel 5. It is also important, in accordance with the invention, to avoid generating 
a large number of threshold crossings associated with the edge of the panel and the background since this 
would increase the processing load on the microprocessor which receives the hits from the FEATUREMAX 

40 board. Therefore, in accordance with the invention, threshold processing is restricted to processing only pixels 
on the board. In the exemplary system, only pixels more than 0.25 inches from the edges of the panel are pro- 
cessed. This is not a drawback since generally components are not placed closer than one quarter inch from 
the edge of a printed circuit board in any event. 

The scheme for utilizing the FEATUREMAX board with a frame size of 512 by 483 pixels to inspect large 

45 panels is illustrated in Figure 4. The array 77 stored in the Capture ROI boards 59 is 2048 pixels wide and covers 
an area 26.6 inches in width W thereby providing a resolution of 0.013 inches at 77 pixels per inch. The array 
77 is also 2048 pixels high representing a length L of 24.6 inches along the conveyor thereby providing a re- 
solution of 0.012 inches for each pixel at 83.3 pixels per inch. In the exemplary system, the largest panels 
inspected are 24 inches by 21 inches as represented by the outline 79 superimposed on the capture ROI array 

so 77. The smallest panel inspected by the exemplary system is 12 inches by 12 inches as represented by the 
outline 81 in Figure 4. The feeder system left justifies the panels as they are loaded onto the conveyor 9a so 
that they are all spaced a fixed distance from the left margin in the Capture ROI array 77. In addition, the 
optosensor and encoder are used to initiate capture of the data as described so that the data for all of the panels 
starts at a fixed distance from the top of the Capture ROI array. 

55 Even for the smallest panel to be inspected by the exemplary system, the number of pixels needed to cover 
the panel exceeds the frame size of the FEATUREMAX board. In accordance with the invention, successive 
sections, such as the sections 83, 85, 87 and 89, each equal in size to the 512 by 483 pixel frame size of the 
FEATUREMAX board are successively transferred to the FEATUREMAX board for processing. As previously 
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mentioned, and in order to reduce the processing required, only sections more than one quarter inch dimension 
B in Figure 4, from the edges of the panel image are transferred to the FEATURE MAX board. The successive 
sections, each equal to a frame of the FEATURE MAX board, are transferred to the FEATUREMAX board until 
the entire panel has been processed. When less than a full frame remains to be processed at the right and at 

5 the bottom, the frame is moved left or up to within one quarter inch of the right and/or the bottom respectively 
of the image, thus, overlapping a previously processed area as shown at the left side 91 of the section 85, the 
upper portion 93 of the section 87 and by the portions 95 and 97 of the section 89. Any resultant redundancy 
is eliminated as will be seen in the processing of the data. -Since the data is generally extremely sparse, this 
does not burden the processing. 

10 In inspecting panels for defects, the digitized video data are entered into the Capture ROI boards 57a and 
57b in real time as the panels move past the optical data acquisition system lines of sight This data acquisition 
is effected independently for the two channels, channel zero and channel one, inspecting opposite sides of the 
panels at the inspection stations 3a and 3b, respectively. 

A flow chart for a suitable program implemented by the microprocessor 39 to process one channel for a 

15 panel is illustrated in Figure 5. When, as indicated at 99 in the flow chart, an interrupt from the optosensor 17a 
or 17b reports that a panel is approaching the camera line of sight, the panel is allocated a descriptor at 101 
for identification. The timer in the input/output timer board 47 is then set for the first speed measurement and 
the count generated by the encoder 35a or 35b is cleared all as indicated at 103. When the timer times out, 
the encoder is read and used to calculate the speed of the panel on the conveyor as indicated at 1 05. The timer 

20 is then set for the calculated time at which the panel will reach the camera line of sight as indicated at 107. 
When this interval has expired, a signal is sent to MAX-SCAN, 55a or 55b, to initiate image capture as indicated 
at 109. The capture ROI board for the channel 57a or 57b generates an interrupt when the image capture has 
been completed. Upon receiving this interrupt at 1 1 1, the captured image is made available for processing by 
placing it in a queue at 1 13. As will be observed, one FEATUREMAX board processes the images generated 

25 by both of the channels. When FEATUREMAX is available, FEATUREMAX processing of the image is initiated 
at 1 1 5. As discussed, the FEATUREMAX board generates an array of hit tables indicating the number of pixels 
having gray-scale values which are below selected thresholds, the addresses of those pixels and the lowest 
threshold breached. Upon the completion of processing by FEATUREMAX, the hit descriptor identifying the 
array of tables of hits for the processed image is queued for processing at 1 17. The results of this processing 

30 of the hits, such an indication that the panel is to be rejected, is received at 1 19 and the final output for the 
channel is set at 121. For instance, if the indication received from the processing subsystem is that the panel 
is to be rejected, the output is set to cause the panel to be diverted downstream. This completion of processing 
of the channel is indicated 1 23 and the program waits at 1 25 for another panel to approach the inspection sta- 
tion. 

35 When FEATUREMAX processing is initiated at 1 15 in Figure 5, the routine shown in Figure 6 is run. When 
this routine is called at 127, the origin and size of each of the sections of the pixel array stored in the capture 
ROI memory and any overlaps are calculated at 129 and a state flag is set to REQUEST at 131. The system 
then waits for an interrupt from the Capture ROI board as indicated at 1 33. The Capture ROI board generates 
this interrupt at the beginning of the vertical blanking interval. Since FEATUREMAX processing is just being 

40 initiated, hit processing will not be in progress when checked at 135. Since there is a request at 1 37, the state 
flag is set to start at 139, so that when the state is checked at 141 , the program will set the pixel index to the 
initial index of the frame at 143, and set the thresholds at 145. As mentioned, the FEATUREMAX board is pro- 
vided with four thresholds against which it compares the gray scale value for each pixel in the frame being pro- 
cessed. With the thresholds set, the input channel to be processed is set at 147 and the FEATUREMAX board 

45 is started at 149. The FEATUREMAX board then compares the gray scale value for each of the pixels in the 
frame to the four thresholds and stores the hits, that is the pixels having a gray-scale value below any of the 
thresholds, in a hit table. The state flag is then set to Ml DDLE at 1 51 and the program waits for the next interrupt. 
Upon receiving the next interrupt from the Capture ROI board, the pixel index is then set to the initial pixel of 
the next frame at 153 and the next frame is processed by FEATUREMAX at 155. While this is occurring, the 

so hit count for the previous frame and the hits for the previous frame are transmitted to the microprocessor at 
157 and 159, respectively. If there are more frames to be processed as determined at 161, the state remains 
MIDDLE and the additional frames are processed in a similar manner. When all of the frames required to pro- 
cess the larger array stored in the capture ROI for the panel have been processed by FEATUREMAX, the state 
is set to END at 163. Following the next Capture ROI interrupt FEATUREMAX operation is terminated at 165. 

55 The hit count for the last board is transmitted at 1 67 and the hits for the last frame are transmitted at 1 69 before 
the state is set to DONE at 171. 

The flow chart for a routine for u nloading hits from the FEATU REMAX board to the microprocessor as called 
for at 1 09 and 1 69 in the flow chart of Figure 6 is illustrated in Figure 7. This unloading of the x and y coordinates 
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of the hits and the thresholds breached must be implemented very rapidly and is accomplished during the ver- 
tical blanking interval of the Capture ROI board from which the data is being processed. As indicated at 173 in 
Figure 7, a POINTER is set to the location in the memory of the microprocessor at which storing of the hit table 
is to be initiated, and a parameter TOTAL, which tracks the number of hits transferred to computer memory, is 

5 initialized to zero. A loop is then entered at 175 to transmit the hit table, with the number of hits to be transferred 
equal to the count generated by the FEATUREMAX board of the number of hits recorded in the hit table for 
the frame being processed. Until that total number of hits has been transmitted, an entry from the hit table is 
fetched at 177. In order to eliminate double inclusion of hits for overlapping sections of the Capture ROI pixel 
array, the microprocessor in defining the frame sized sections of the Capture ROI to be successively processed 

10 by the FEATUREMAX board, specifies minimum x and y coordinates of the frame to eliminate the overlap. 
Accordingly, if the x and y coordinates of the hit under consideration are not in the overlap area as determined 
at 179 and 181 , the data for the hit is copied to the computer memory at 183. The parameter TOTAL and the 
POINTER are then incremented at 185. The video active, VA, bit generated by the Capture ROI board is then 
checked at 1 87. If this video active bit indicates that the Capture ROI board is still in the vertical blanking interval, 

15 than COUNTER is decremented at 191 and the additional entries in the hit table are transmitted to microproces- 
sor memory in a similar manner. When all of the hits in the hit table have been transmitted as determined at 
175, or the Capture ROI has begun to transmit a new frame of data to the filter as determined at 189, then a 
parameter ACTUAL indicating the number of hits actually transferred to the microcomputer is set equal to total 
at 193. If a RESTART flag indicating that there are additional frames of the Capture ROI image to be processed 

20 is set, as determined at 195, then a START NEW FRAME flag is set at 197 before the routine returns to the 
calling program at 199. 

A flow chart for image and hit table processing by the microprocessor 39 is illustrated in Figure 8. With the 
system in the inspection mode as indicated 201, if a break key is depressed as determined at 203, indicating 
that the operator is interfacing with the system, the inspection mode is exited at 205 for the menu mode. If the 

25 operator is not interfacing with the system, a determination is made at 207 if an image is available for processing. 
When an image is available, the edge of the panel within the image is located at 209. This location of the panel 
edge is then used to plot the expected board outline on the MAX-GRAPH board 71, which it will be recalled, 
is used to geneiate the display on the monitor 29. If the operator has selected plot enable as determined at 
213, the measured board outline can also be plotted on MAX-GRAPH as indicated at 21 5. This measured board 

30 outline is established by routine which locates the edges of the board from the gray-scale values of the image 
stored in Capture ROI. 

Next, if a hit table has been made available in the queue as determined at 217, the total number of hits for 
the panel is determined at 219. This count is the count of the number of pixels for which the gray-scale value 
exceeded any of the four thresholds. If this total hit count exceeds a preselected value, as determined at 221 , 

35 the panel is rejected. Otherwise, proximate hits are associated into blobs and the number of hits in each blob 
is calculated as determined at 223. The size of the largest blob is then determined at 225 and if this largest 
blob size exceeds a threshold, the reject flag is set at 229. The hits are then plotted on the MAX-GRAPH 
superimposed on the outline of the panel as indicated at 231 . The memory is then released at 233 for receiving 
new hit table data from the FEATUREMAX as indicated at 233 and the results are logged on the printer at 235. 

40 This loop is repeated as long as the microprocessor remains in the inspection mode. 

The flow chart for the routine for associating proximate hits into blobs and calculating the number of hits 
in each blob is illustrated in Figure 9a and 9b. Data structures used by this routine are illustrated in Figures 10 
through 12. Figure 10 illustrates the hit array 237 in which the hits are sequentially numbered and their x and 
y coordinates are stored. Up to a maximum number of hits can be stored in the array. This max number of hits 

45 corresponds to the maximum number of pixels exceeding a threshold that can be tolerated. The blob tag array 
239 of Figure 1 1 indicates the identification of a blob, if any, with which each of the numbered hits has been 
associated. The blob array 241 shown in Figure 12 lists for each identified blob the number of hits associated 
with the blob and the identification of "which" hits are associated with the blob. The "which" entry 243 has been 
expanded in Figure 12 to show that it can include the identification of a number of hits up to a maximum size 

so for the blob. If any panel has a blob with the maximum number of hits in it, the panel is rejected. 

Returning to Figures 9a and 9b, a parameter INDEX which points to a hit of interest is initialized to two at 
245. This starts the processing with the second set of x, y coordinates in the hit array. Another pointer identified 
as POINTER is set to one less than INDEX as indicated at 247. Next, the distance D y from the y value of the 
hit pointed to by INDEX to the y value of the hit pointed to by POINTER is calculated in 249. If this distance, 

55 D y is not greater than a maximum allowable distance between hits associated into a blob, D 8p3C j g g as determined 
at 251, then the absolute value of the distance, D x for the same two hits is calculated at 253. If Dx is less than 
or equal to D s?aciI ^ as determined at 255, then a flag CONNECTED is set to true at 257 indicating that the hits 
pointed to by INDEX and POINTER are close enough to be associated in a blob. If the x spacing is too great 
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to associate the hits into a blob, POINTER is decremented at 259 to move backward through the hit table and 
the proximity test is repeated until the hit of interest has been evaluated with respect to all of the earlier hits in 
the hit table as determined at 261 . 

When the hit selected by the pointer INDEX has been evaluated with regard to all of the previously entered 

5 hits in the hit table and the hit was within the parameters set for proximity as indicated by the true condition of 
the CONNECTED flag at 263 on figure 9b, the blob tag for the blob with which the hit is associated is entered 
at 265 in the blob tag array. The "how many* value for that blob in the blob array is then incremented at 267, 
and the hit indicated by INDEX is entered into the "which" array at the position pointed at by "how many" as 
indicated at 269. The pointer INDEX is then incremented by one at 271. If the hit was not close enough to pre- 

10 viously enter hits to be associated with any of them into a blob, so that the CONNECTED, flag is not set at 263, 
then a new blob is indicated at 273 in the blob tag table for the hit pointed to by INDEX and the cooresponding 
"how many" entry in the blob array is initialized to one at 275. The hit pointed to by INDEX is then copied into 
the "which" array at the first position for the new blob as indicated at 277. NEW BLOB is then incremented by 
one at 279 for the next blob and INDEX is incremented at 271 to select the next hit in the hit array. If all of the 

15 hits in the hit table has not yet been evaluated as determined at 281 , the program loops back to 247, otherwise 
the routine is exited at 283. 

The vision systems of the embodiments described and illustrated above have enabled vision processing 
equipment with limited area coverage to be used to inspect panels of larger area. Furthermore, the data indi- 
cative of selected defects can be reduced to reduce processing time as well as the capacity required for the 

20 vision systems to perform the inspection on line. 
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40 Claims 



1. A vision system for inspecting a part having a substantially flat reflective surface, characterized in that said 
system comprises an optical data acquisition system (19a,19b), comprising a light source (21a, 21b) for 
directing light at the substantially reflective surface; and a camera (23a, 24b) for generating pixel gray- 

45 scale values for light from said light source means reflected from said flat reflective surface; and a vision 

processor (25) comprising a storage system (57a, 57b) for storing a pixel array of gray-scale values gen- 
erated by said camera (23a, 23b); an image processing board (67) identifying pixels in said pixel array 
with predetermined gray-scale values; and a processor (39) for evaluating said identified pixels and 
generating an output based thereon. 

50 

2. A system according to claim 1 , characterized in that the vision processor (25) includes a transfer system 
(61 , 63, 65) for successively transferring to said image processing board (67) for processing, gray-scale 
values for successive frame sized sections of the pixel array of the storage system (57a, 57b) until the 
gray-scale values for all of the pixels of said pixel array have been processed by the image processing 

55 board (67); and a processor (39) for evaluating said identified pixels and generating an output based the- 

reon. 

3. A system according to claim 2, characterized in that the transfer system (61 , 63, 65) is adapted to transfer 
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to the image processing board (67) gray-scale values for only those sections of the pixel array of the stor- 
age module (57a, 57b) which corresponds to a portion of the substantially flat reflective surface which is 
more than a predetermined distance from the edges of the part. 

A system according to claim 2 or 3, characterized in that the transfer system (61, 63, 65) is adapted to 
overlap the frame-sized sections of the pixel array of the storage system (57a, 57b) transferred to the image 
processing board when the portion of the substantially flat reflective surface is not a whole multiple of said 
frame size of the image processing board (67), as well as to eliminate double inclusion of identified pixels 
in overlapping portions of the pixel array processed by the image processing board. 

A system according to claim 1 , 2, 3 or 4, characterized in that defects in the substantially flat reflective 
surface have a lower reflectivity than said surface, and in that the image processing board (67) identifies 
pixels having gray-scale values which fall below a selected threshold. 

A system according to any of claims 1 to 5, characterized in that the vision processor includes a filter (63) 
which increases contrast in said gray-scale values transferred to the image processing board (67) and 
reduces the number of pixels identified by said image processing board to reduce processing required. 

A system according to any of claims 1 to 6 which is adapted for inspecting a panel having first and second 
substantially flat reflective surfaces characterized in that the optical data acquisition system comprises a 
first component (1 9a) operable to generate a first set of pixel gray-scale values for light reflected from the 
first substantially flat reflective surface of said panel and a second component (19b) operable to generate 
a second set of gray-scale values for light reflected from the second flat reflective surface of said panel, 
and in that the storage system of the vision processor (25) includes a first storage module (57a) for storing 
a first pixel array of the first set of pixel gray-scale values and a second storage module (57b) for storing 
a second pixel array of the second set of pixel gray-scale values, and in that the transfer system (61 , 63, 
65) is operable to successively transfer frame sized sections of one pixel array to the image processing 
board (67) to identify pixels with gray-scale values below the selected threshold and then to successively 
transfer frame sized sections of the other pixel array to said image processing board (67) for identifying 
pixels having gray-scale values below said selected threshold, and in that the processor evaluates the 
identified pixels from both said first and second pixel arrays. 

A system according to any of claims 1 to 7, characterized in that the processor (39) is adapted to associate 
identified pixels within a predetermined number of pixels of another identified pixel into a blob and generate 
a reject signal when the number of pixels in a blob exceeds a predetermined number. 

A system according to claim 8 characterized in that the image processing board (67) compares gray-scale 
values to the pixels with multiple thresholds and identifies hits which breach each threshold and in that 
the processor (39) weights hits included in a blob dependent upon the thresholds breached and generates 
a reject signal based on the weighted number of pixels in a blob. 

A system according to any of claims 1 to 8, characterized in that the image processing board (67) has a 
frame size covering an array of pixels smaller than the pixel array stored in the storage system (57a, 57b). 



14 



EP 0 465 233 A2 




15 



EP 0465 233 A2 




16 



EP 0465 233 A2 




17 



EP 0 465 233 A2 




EP 0 465 233 A2 



( WAITING ^ 25 
I 



-119 



INTERRUPT FROM OPTO-SENSOR 



I 



ALLOCATE A PANEL DESCRIPTOR 



I 



.99 
J01 



WAIT TO GET INDICATIONS 
FROM PROCESSING SUBSYSTEM 



SET TIMER FOR FIRST SPEED 
MEASUREMENT AND CLEAR 
THE ENCODER 



-103 



TIMER INTERRUPT - READ ENCODER 
AND CALCULATE SPEED OF PANEL 



105 



SET TIMER FOR START OF 
IMAGE CAPTURE 



I 



.107 



TIMER INTERRUPT - START 
MAXSCAN TO CAPTURE IMAGE 



I 



109 



WRITE INTERRUPT FROM 
CAPTURE ROI 



I 



MAKE IMAGE AVAILABLE FOR 
PROCESSING VIA QUEUE . 



I 



J11 



J13 



START FEATUREMAX 
PROCESSING 



I 



.115 



RELEASE HIT DESCRIPTOR 
TO QUEUE FOR PROCESSING 



.117 



121 



I 



SET FINAL OUTPUT 
INDICATION FOR CHANNEL 




I 



END OF CHANNEL 
PROCESSING 



FIG. 5 



19 



EP 0 465 233 A2 



c 



IMAGE CAPTURE AND 
HIT PROCESSING REQUIRED 



1 



127 



CALCULATE ORIGIN AND 
SIZE OF EACH FRAME 



FIG. 6 



SET STATE TO REQUEST 



129 



-131 



WAIT FOR INTERRUPT]-^ 



133 



135 



INTERRUPT 



HIT PROCESSING 
IN PROGRESS 



141 



> 



C 



.137 



NO 



ANY 
REQUEST 

H 




SET STATE TO START 



.139 



STATE = ? 



START 
143 




MIDDLE 



SET PixO OF FRAME 



145 
147. 



I 



SET THRESHOLDS 



I 



155- 



SET PixO OF FRAME 



r 



165 



STOP FEATUREMAX 



\3ET INPUT CHANNEL 



149. 



I 



START FEATUREMAX 



I 



157. 



159- 



STE P TO NEXT FRA ME 



I 



GET HIT COUNT 
1 



■ GET HIT COUNT 
167 / | — 



169 



UNLOAD HITS 



SET STATE TO MIDDLE 



151^ 



161 



UNLOAD HITS 
I 



SET STATE TO DONE 



DONE? 



YES 



163^ 



1 



SET STATE 
TO END 



I 



171 



20 



EP 0 465 233 A2 



i 



PTR = BUFFER 
TOTAL = 0 



173 



175 



COUNTER 



7$* 



FETCH HIT (X,Y) Vl77 



STORE HIT 



TOTAL = TOTAL + 1 
PTR = PTR + 1 

i 



183 



185 



CHECK VA BIT 

~~T~ 

FRAME START ? 



187 



U89 



NO UW 



COUNe = COUNTER - 1 



.191 



193 



ACTUAL = TOTAL 



RESTART 
FLAG 



VJ95 



197 



START NEW FRAME 



Q RETURN ^ 



199 



FIG. 7 



21 



EP 0 465 233 A2 



INSPECTION 
MODE 




BREAK 
KEY 
DEPRESSED 




IMAGE 
AVAILABLE 



LEAVE INSPECTION MODE 
ENTER MENU MODE 



LOCATE PANEL 
EDGE WITHIN 
IMAGE 



205 



211-/ 



PLOT EXPECTED 
BOARD OUTLINE 
ON MAXGRAPH 



213 




HIT 
TABLE 
AVAILABLE 




COUNT HITS 



PLOT 
ENABLED 



215 



> 



PLOT MEASURED 
BOARD OUTLINE 
ON MAXGRAPH 



FIG. 8 



# HITS 
> THRESHOLD 




221 



ASSOCIATE PROXIMATE 
HITS INTO BLOBS AND 
CALCULATE NUMBER OF 
HITS IN EACH BLOB 
i 



223 



DETERMINE 
LARGEST BLOB 




BLOB SIZE 
EXCEEDS 
THRESHOLD 




SET REJECT 
FLAG 



\229 



PLOT HITS 
ON MAXGRAPH 



V231 



RELEASE 
MEMORY 



233 



LOG RESULTS 
ON PRINTER 
1 



"V235 



22 



EP 0 465 233 A2 



( ENTER y\2U 




INITIALIZE 
POINTER TO 
ONE < INOEX 



.247 



I 



CALCULATE DISTANCE 

Dy FROM INDEX 
Y TO Y OF HIT POINTED 
AT BY POINTER 



I 



.249 



Of 

> D SPAC1NG 



251 




CALCULATE ABSOLUTE 
VALUE OF D1STACE 
DX FOR SAME 2 HITS 



"\253 




I 



$ "spacing 





DECREMENT 




POINTER 



I 



/ 



259 



CONNECTED 
25 7 ^/| IS TRUE 



POINTER 
AT BEGINNING 
OF TABLE 




FIG. 9 A 

23 



EP 0 465 233 A2 




i 



CONNECTED 
TRUE 



i 



> 



COPY NEW BLOB TO 
BLOB TAG POINTED 
TO BY INDEX TO 
START A NEW BLOB 



~\273 



INITIALIZE 
CORRESPONDING 
HOW MANY TO 1 



275 



COPY INDEX INTO WHICH 
ARRAY AT FIRST POSITION 



I 



267. 



269 



\277 



INCREMENT NEW 
BLOB BY 1 



"\279 



L 



265 



COPY BLOB TAG 
VALUE TO NEW HIT 



INCREMENT HOW 
MANY FOR THAT BLOB 



COPY HIT INDEX INTO 
WHICH ARRAY AT POSITION 
POINTED AT BY HOW MANY 




EP 0 465 233 A2 



FIG. 10 



MAXHir 



FIG. 11 



MAXHIT 



HIT ARRAY 
X Y 



BLOB TAG ARRAY 




BLOB ARRAY 
HOWMANY WHICH 



FIG. 12 



MAXSIZE 




25 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 

BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations-of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 
-OTfaded TEXT OR DRAWING 

□ BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

JZi LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE^) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

VN □ OTHER: . . 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



